System and method for automatically measuring the dimensions of and identifying the type of exterior siding

ABSTRACT

Methods, systems, and computer readable media are disclosed for determining a pixel-to-length ratio between a number of pixels disposed over a predetermined length of a reference object within an image of a siding sample and the predetermined length of the reference object. A first and second distance between respective first and second pairs of points within the image corresponding to respective first and second length measurements of the siding sample are determined, as well as a first and second number of pixels disposed between the first and second pair of points, respectively. Furthermore, the method, system, and computer readable medium disclose determining the first length measurement based on the pixel-to-length ratio and the first number of pixels, determining the second length measurement based on the pixel-to-length ratio and the second number of pixels, and identifying a siding product associated with the first and second length measurements.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/228,617, filed on Mar. 28, 2014 (issued as U.S. Pat. No. 8,885,916 onNov. 11, 2014) which is incorporated herein by reference in itsentirety.

FIELD OF THE DISCLOSURE

The present disclosure relates to siding identification and, moreparticularly, to measuring one or more dimensions of siding using acaptured image and identifying products matching the measurements.

BACKGROUND

After sustaining exterior siding damage to a residence or a commercialbuilding, a client typically files a claim with their insurance company.The insurance company then assigns an agent to investigate the claims todetermine the extent of damage and to provide the client withappropriate compensation. Agents have had long-standing difficultiesidentifying the dimensions of the damaged exterior siding, which oftenresults in repair delays and/or unnecessary expenses. For example, ifthe dimensions of damaged siding are misidentified, incorrectly ordered(and sometimes installed) replacement siding may not match the size ofthe existing siding to the satisfaction of the client. In this case, thecontractor may need to return the replacement siding, reattemptidentification of the damaged siding, and install (or reinstall) newreplacement siding.

Correctly identifying siding dimensions can be an arduous process.Generally, an agent typically locates and removes a section of damagedsiding from the building similar to the damaged siding to obtain aphysical sample. The agent may need to take multiple measurements of thesample to sufficiently identify a type and manufacturer for a potentialreplacement. Using these measurements, the agent tries to identify agiven style of siding (e.g., Dutchlap, clapboard, board and batten,etc.) by a measured face and profile size. This process requires theagent to travel to the site with measurement and/or siding removaltools. Furthermore, training is required for the agent to learndifferent styles of siding and how to properly recognize and measure thedifferent types of faces and/or profiles for each style. Even when thedimensions of the siding are properly measured, properly identifying anavailable replacement siding product that best matches the dimensions ofthe sample is a significant problem.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a siding identification system 10 inaccordance with an exemplary embodiment of the present disclosure;

FIG. 2 is an example screenshot 60 that illustrates an image of sidingsamples 62 and 64 having a different number and size of faces that oneanother;

FIG. 3 is an example screenshot 90 that illustrates an image of a sidingsample 91 having a face size shorter than a profile size;

FIG. 4 is an example screenshot 110 that illustrates an image of asiding sample 112 with a profile size corresponding to a type of siding;

FIG. 5 is an example screenshot 130 that illustrates an image of sidingsample 91 and a reference object in accordance with an exemplaryembodiment of the present disclosure;

FIG. 6A is an example screenshot 150 that illustrates an image of asiding sample captured from a severe angle in accordance with anexemplary embodiment of the present disclosure;

FIG. 6B is an example screenshot 160 that illustrates an image of asiding sample after performing image rectification on the image of thesiding sample in FIG. 6A in accordance with an exemplary embodiment ofthe present disclosure; and

FIG. 7 illustrates a siding identification method 700 in accordance withan exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a siding identification system 10 inaccordance with an exemplary embodiment of the present disclosure.Siding identification system 10 includes a client device 14 configuredto connect to a siding identification engine 12 via a network 16.Network 16 may include any appropriate combination of wired and/orwireless communication networks. For example, networks 16 may includeany combination of a local area network (LAN), a metropolitan areanetworks (MAN), a wide area network (WAN), and may facilitate aconnection to the Internet. To provide further examples, network 16 maybe include wired telephone and cable hardware, satellite, cellular phonecommunication networks, etc.

Client device 14 includes an image capture device 42, a user interface36, a central processing unit (CPU) 32, a graphics processing unit (GPU)34, a memory 44, a display 38, and a communication module 40. In anembodiment, client device 14 is implemented as a user equipment (UE)such as a mobile device, a computer, laptop, tablet, desktop, or anyother suitable type of computing device.

Image capture device 42 is configured to capture an image in a digitalformat having any suitable number of pixels. Image capture device 42 maybe implemented as a camera coupled to client device 14. Although imagecapture device 42 is illustrated in FIG. 1 as internal to client device14, embodiments include image capture device 42 being internal orexternal to client device 14. For example, image capture device may beimplemented as a webcam coupled to client device 14 and configured tocommunicate with client device 14. Furthermore, image capture device 42may be implemented as a digital camera configured to transfer digitalimages to client device 14 and/or to siding identification engine 12.Such transfers may occur via a cable, a wireless transmission, network16, and/or a physical memory card device transfer (e.g., SD Card, Flashcard, etc.), for example.

User interface 36 is configured to allow a user to interact with clientdevice 14. For example, user interface 36 may include a user-inputdevice such as an interactive portion of display 38 (e.g., a “soft”keyboard displayed on display 38), an external hardware keyboardconfigured to communicate with client device 14 via a wired or awireless connection (e.g., a Bluetooth keyboard), an external mouse, orany other suitable user-input device.

CPU 32 and/or GPU 34 are configured to communicate with memory 44 tostore to and read data from memory 44. In accordance with variousembodiments, memory 44 is a computer-readable non-transitory storagedevice that may include any combination of volatile (e.g., a randomaccess memory (RAM), or a non-volatile memory (e.g., battery-backed RAM,FLASH, etc.).

Memory 44 is configured to store instructions executable on the CPU 32and/or the GPU 34. These instructions may include machine readableinstructions that, when executed by CPU 32 and/or GPU 34, cause the CPU32 and/or GPU 34 to perform various acts.

Length selection module 50 is a portion of memory 44 configured to storeinstructions, that when executed by CPU 32 and/or GPU 34, cause CPU 32and/or GPU 34 to enable user interface 36 to interact with a user. Forexample, executable instructions may enable user interface to displayone or more prompts to a user, and/or accept user input, such as lengthmeasurements within an image of a siding sample. In an embodiment,instructions stored in length selection module 50 enable a user toselect a particular length measurement within the image. For example, auser may utilize user interface 36 to specify a length selection byselecting two points within the image that denote a desired measurementlength. To provide another example, a user may utilize user interface 36to click, hold, or drag a cursor to define a desired measurement length.

Siding identification engine 12 includes CPU 18 and memory 20. Sidingidentification engine 12 is configured to communicate with client device14 via network 16. Siding identification engine 12 is coupled to sidingproduct reference database 30 and is configured to communicate withsiding product reference database 30 to store to and read data fromsiding product reference database 30. Siding identification engine 12 isconfigured to receive one or more measurements from client device 14 andto identify one or more siding products based on the one or moremeasurements by accessing matching measurements from siding productreference database 30. Siding identification engine 12 is configured tosend the identified one or more matching siding products to client 14.

Siding identification engine 12 may be implemented as any suitablecomputing device. In various embodiments, siding identification engine12 is implemented within a web server, a laptop computer, a tabletcomputer, a smartphone, etc. In various embodiment, sidingidentification engine 12 is implemented as an integral part of clientdevice 14, separate from client device 14, and external to client device14. In accordance with embodiments in which siding identification engine12 is implemented as an integral part of client device 14, CPU 32 maycommunicate with siding identification engine 12 (and siding producereference database 30) and network 16 is not needed. Further inaccordance with such an embodiment, CPU 32 and/or memory 44 may have astructure similar to, and provide the same functionality as, CPU 18and/or memory 20, respectively.

CPU 18 is configured to communicate with memory 20 to store to and readdata from memory 44. In accordance with various embodiments, memory 20is a computer-readable non-transitory storage device that may includeany combination of volatile (e.g., a random access memory (RAM), ornon-volatile memory (e.g., battery-backed RAM, FLASH, etc.). Memory 20is configured to store instructions executable by CPU 18. Theseinstructions may include machine readable instructions that, whenexecuted by CPU 32, cause CPU 18 to perform various acts.

Siding query module 22 and desired length data 29 are portions of memory20 configured to store instructions executable by CPU 18. Desired lengthdata 29 is a portion of memory 20 on which siding query module 22operates. Siding query module 22 includes a measurement module 24 that,when executed by CPU 18, causes CPU 18 to calculate a length between twouser-specified points, for example, in terms of pixels or lengthmeasurement units. In an exemplary embodiment, memory 20 includes areference object detection module 26 configured to store instructionsthat, when executed by CPU 18, cause CPU 18 to detect a reference objectwithin an image of a siding sample. In an embodiment, memory 20 includesan image rectification module 28 that, when executed by CPU 18, causesCPU 18 to adjust a perspective of an image and transform the image sothat the image of the siding sample appears to captured from a forwardfacing perspective. This embodiment may be implemented, for example, ifthe image of the siding was captured at an angle deviating from astraight on perspective.

In accordance with various embodiments, any of siding query module 22,reference object detection module 26, and/or image rectification module28 operates as a separately executable software application, a pluginthat extends the functionality of another software application such as aweb browser, an application programming interface (API) invokable by asoftware application, etc. The instructions included within any ofsiding query module 22, reference object detection module 26, and/orimage rectification module 28 may be compiled and executable on the CPU18 directly, or not compiled and interpreted by the CPU 18 on a runtimebasis.

Siding identification engine 12 is configured to access siding productreference database 30 to provide siding product data to siding querymodule 22. Siding product reference database 30 may store measurementdata, such as face size or profile size measurements, for individualsiding products that represent potential replacement candidates for thesiding sample. The siding product data may include not only measurementdata of a particular siding product, but also manufacturer information,product information (e.g., a product number, a product color, a productname, a product price, etc.) In various embodiments, siding productreference database 30 is implemented within client device 14 or sidingidentification engine 12, separate from client device 14 or sidingidentification engine 12, or external to client device 14 or sidingidentification engine 12.

Although illustrated as a single engine in FIG. 1, in variousembodiments, siding identification engine 12 may consist of any numberor group of one or more siding identification engines. In accordancewith such embodiments, each siding identification engine may be equippedwith one or more CPUs and configured to operate independently of theother siding identification engines. Siding identification enginesoperating as a group may process requests from client device 14individually (e.g., based on their availability) and/or concurrently(e.g., parallel processing). Siding identification engines operating asa group may process requests from the client device 14 in a prioritizedand/or distributed manner. For example, an operation associated withprocessing a request may be performed on one siding identificationengine while another operation associated with processing the samerequest (or a different request) is performed on another sidingidentification engine.

FIG. 2 is an example screenshot 60 that illustrates an image of sidingsamples 62 and 64 having a different number and size of faces that oneanother. Example screenshot 60 illustrates images of siding samples 62and 64 that have a similar style but include different number of faces.For example, siding sample 62 is a double clapboard style that includesa nail hem 72 and two faces 66, 68 that occupy an entire width 70 ofsiding sample 62. Similarly, siding sample 64 is a triple clapboardstyle and includes a nail hem 82 and three faces 74, 76, 78 that occupythe width 80 of siding sample 64. In this example, width 70 of sidingsample 62 is shorter than width 80 of siding sample 64 to illustratethat widths of siding may vary depending on the style, manufacturer,material, etc.

Generally speaking, a face of a siding sample is a uniform, flat portionof the siding, which may include multiple faces. For example, as shownin FIG. 2, the face of the siding includes a portion of the siding thatis substantially parallel with a mounting surface to which the siding isaffixed. In other words, the face of a siding sample is the portion isthat is substantially plumb or substantially vertical when the siding isinstalled. The double clapboard style of siding sample 62 includes aface length 67 having the same length as profile length 69. Likewise,the triple clapboard style of siding sample 64 includes face length 77having the same length as profile length 79.

FIG. 3 is an example screenshot 90 that illustrates an image of a sidingsample 91 having a face size shorter than a profile size. In contrast toFIG. 2, siding sample 91 illustrates a style having different face andprofile lengths. Other siding styles may have other dimensions that aredifferent sizes from one another, and in different proportions. Forexample, as shown in FIG. 3, a siding sample 91 of a Dutch lap styleincludes two profiles 94, 98, two faces 92, 96, and a nail hem 100.Faces 92 and 96 are both flush or parallel to a mounting surface.Moreover, the remaining portion 93 of profile 94 is not flush to themounting surface. Rather, portion 93 slopes back at an angle toward themounting surface. Thus, in siding sample 91 of a Dutch lap style, facelength 102 of the face 92 is a different length than profile length 104of the profile 94. That is, profile length 104 of profile 94 whollyencompasses face length 102 of face 92. A Dutch lap style siding mayinclude any number of different ratios of face length 102 to profilelength 104.

According to industry nomenclature, the term “profile” generally refersto any contours formed in siding to create a distinctive style. Forinstance, various styles of siding or exterior structure coverings mayinclude board and batten, clapboard, double beaded, traditional lap,Dutch lap, full bead, half bead, half rounds, logs, masonry, panel,shakes, soffit-beaded, soffit-flat panels, soffit-U groove soffit,soffit-V groove soffit, vertical panels, horizontal panels, scallops,etc. The profile of a siding style may be repeated throughout portion ofthe design over the width of the sample. For example, as illustrated inFIG. 2, siding sample 62, a double clapboard style, repeats the sameprofile twice and siding sample 64, a triple clapboard style, repeatsthe same profile three times. Furthermore, the term “face” generallyrefers to any generally flat portion of a piece of siding. But the term“face” may also refer to any flat portion of a piece of siding that runsparallel with the siding mounting surface that may include a batten orany other flat portion of a style of siding in addition to a generallyflat portion of a piece of siding.

FIG. 4 is an example screenshot 110 that illustrates an image of asiding sample 112 with a profile size corresponding to a type of siding.Siding sample 112 illustrates a board and batten style and includes abatten portion 114, a profile portion 116, and a nail hem 118. Battenportion 114 is generally a raised portion relative to the rest ofprofile portion 116 of siding sample 112. Moreover, batten length 122 isgenerally shorter than profile length 120. Board and batten style siding112 may include any number of different ratios of batten length 122 toprofile length 120.

To determine various lengths of different portions of a particularsiding sample, various embodiments of siding identification system 10utilize a reference object within the image of the siding sample.

FIG. 5 is an example screenshot 130 that illustrates an image of sidingsample 91 and a reference object 134 in accordance with an exemplaryembodiment of the present disclosure. For example, a user (e.g., anagent) may affix or place the reference object onto a flat portion of asiding sample so that the reference object rests on a surface of thesiding sample prior to capturing an image of it.

Examples of reference objects may include peel-and-stick decals, paper,cardboard, and/or any other suitable material cutouts having any numberof sizes, colors, and/or patterns. Reference object 134 has apredetermined height 136 and width 137 in standard measurement unitssuch as inches, centimeters, etc. For example, reference object 134 maybe a square shape and have a height 136 measuring 3 inches and a width137 measuring 3 inches. As will be appreciated by those of skill in therelevant art(s), reference object 134 may be implemented using any typeof shape, such as a rectangle, a triangle, a hexagon, etc., that may beutilized in algorithmic pattern detection. To provide an example,reference object 134 may include a checkerboard or chessboard pattern asshown in FIG. 5. To provider another example, reference object mayinclude a border trim of a first color and a fill inside the border trimof a second color, such as a white border with a black fill. Suchcolored patterns may facilitate detection of reference object 134 on awide variety of different colored siding backgrounds. Reference object134 may include any suitable pattern that is machine recognizable viaalgorithmic image detection, line detection, and/or any other suitableimage processing or detection techniques.

Since reference object 134 may be implemented with lightweight materialssuch as decals, an agent can conveniently carry a variety of differentsized reference objects best suited to measure different lengths withina siding sample image. In other words, an agent could use smallerreference objects for siding having smaller measurement lengths andlarger reference objects for siding having larger measurement lengths.Since the reference objects are lightweight and easy to transport, anagent can carry an appropriate number of different sized referenceobjects without being overly encumbered in doing so.

In accordance with an embodiment, siding identification system 10 usesone or more of image detection, feature detection, and/or patterndetection techniques to detect reference object 134 within the image. Aswill be appreciated by those of ordinary skill in the relevant art(s),siding identification system 10 may implement any combination of a SIFT(Scale-invariant feature transform), a SURF (Speeded Up RobustFeatures), a Hough transform, and/or any other pattern recognition, edgedetection, corner detection, blob detection, ridge detection, featureextraction, or image processing technique to detect reference object134. Siding identification system 10 determines the height and/or widthof reference object 134 in terms of any suitable units once referenceobject 134 is detected and its boundaries are determined. In accordancewith an embodiment, siding identification system 10 determines theheight and/or width of reference object 134 in terms of pixels.

For example, siding identification system 10 could detect the chessboardpattern of reference object 134 and determine that the height 136 ofreference object 134 in the image is 500 pixels. Likewise, depending onthe shape of the reference object 134, siding identification system 10may also determine width 137 of reference object 134 in terms of pixels(e.g., 500 pixels if reference object 134 is a square shape). Once theheight and/or width of reference object 134 in pixels is determined,siding identification system 10 may use the known height and/or width ofreference object 134 (i.e., the physical dimensions) to calculate aratio between the number of pixels (i.e., 500) and the known physicallength corresponding to the pixel measurement. This ratio may bereferred to as a pixel-to-length ratio. For example, if reference object134 has physical measurements of 2 inches×2 inches, then thepixel-to-length ratio in the previous example would be

$\left( \frac{500\mspace{14mu}{pixels}}{2\mspace{14mu}{inches}} \right) = {250\mspace{14mu}\text{pixels-per-inch.}}$

In an embodiment, siding identification system 10 uses thispixel-to-length ratio to calculate any other desired measurement of thesiding sample within the image using a number of pixels associated witha desired length to be measured. Further in accordance with thisexample, once the pixel-to-length ratio is determined, a user selects apair of points within the image represented by screenshot 130 such thatthe length spanning between the pair of points is face length 138.Siding identification system 10 then determines the number of pixelsassociated with face length 138, such as 437 pixels, for example. Usinga pixel-to-length ratio of 250, siding identification system 10determines that the physical length of face length 138 in this exampleis

$\left( \frac{437\mspace{14mu}{pixels}}{250\mspace{14mu}\text{pixels/inch}} \right) = {1.748\mspace{14mu}{{inches}.}}$Siding identification system 10 may repeat these calculations to obtainany number of other measurements within the image, such as profilelength 140, etc.

The image represented by screenshot 130 shows reference object 134affixed to face 92 of siding sample 91, although reference object 134may be positioned at any location on siding sample 91. But it may bepreferable to execute additional processing steps to accuratelydetermine the pixel-to-length ratio if reference object 134 is notaligned parallel with a portion of the siding sample and/or the image ofreference object 134 is not captured from a straight on perspective. Aswill be appreciated by those of ordinary skill in the relevant art(s),some image detection or line detection techniques require straight linesor utilize only horizontal or vertical lines. As a result, it ispreferable in some embodiments of the present disclosure that images ofsample siding are measured at a substantially straight on shotperspective and/or edges of reference object 134 and edges of sidingsample 91 be aligned with one another and the horizontal-vertical plane.Various embodiments of the present disclosure include correcting animage to compensate for perspectives and/or alignments deviating fromthese preferences.

Various embodiments of the present disclosure include different levelsof automation to obtain one more siding measurements and to identify oneor more matching siding products. For example, once an image of thesiding sample and reference object are captured, siding identification10 may implement an automatic system or a semi-automatic system tomeasure portions of the siding sample and to identify matching sidingproducts.

Embodiments of siding identification system 10 implementing automaticsiding product identification first determines the type of siding viaimage recognition techniques and/or user entry. Once the type of sidingis determined and a user enters the known physical lengths associatedwith the reference object, siding identification system 10 identifiesedges of reference object 134 and determines a pixel-to-length ratiobased on the detected edges and physical lengths entered by the user.Siding identification system 10 may utilize the face size and profilescorresponding to the identified style to identify face and/or profilemeasurements via image recognition. Siding identification system 10 mayrecognize edges in the image and determine a number of pixels associatedwith face length 92 based on information obtained from the correspondingsiding style.

For example, siding identification system 10 may compare one or morestored reference images representative of a determined (or input) sidingstyle to the captured image and determine which portion of the imagecorresponds to one or more desired length measurements (e.g., face andprofile lengths). Once the appropriate portions of the image to bemeasured are identified, siding identification system 10 may determine anumber of pixels spanning the measurements, a corresponding lengthmeasurement using the pixel-to-length ratio, and then query and returnany matching siding products. In accordance with an embodiment of thepresent disclosure, a user is provided the matching siding productswithout user intervention once a siding image is obtained and thereference object dimensions are determined such that the pixel-to-lengthratio is calculated.

Embodiments of siding identification system 10 implementingsemi-automatic siding product identification may determine a type ofsiding via image recognition techniques and/or user entry. Once the typeof siding is determined and a user enters the known lengths associatedwith the reference object for pixel-to-length calculation, desiredsiding measurements may be queried by the user. Alternatively, sidingidentification system 10 does not acquire a siding type insemi-automatic embodiments and a user may specify only lengthmeasurements for siding identification; however, to better narrowqueried results to provide fewer siding product matches, it ispreferable that a user specify a siding type in such embodiments tonarrow the pool of potential matches.

Using FIG. 5 as an example, a user may utilize user interface 38 todefine a desired line length measurement within the image. For example,a user may move, click, drag, etc., a cursor to specify two pointswithin the image, the line formed between the two points correspondingto the desired length measurement. A user may repeat this any number oftimes to identify any number of length measurements, such as face length92 and profile length 94, for example. Siding identification system 10then determines a number of pixels associated with a correspondinglength measurement and uses the pixel-to-length ratio to determine theirassociated physical lengths.

In accordance with both automatic and semi-automatic embodiments, sidingidentification system 10 utilizes any number of user prompts at variousstages to verify the accuracy of the pixel-to-length ratio, identifiededges and/or length measurements, siding type, siding measurements,and/or identified matching siding products. In accordance with variousembodiments, siding identification system 10 may prompt a user for asiding type, verification of the siding type, to retake the image, toverify measurements identified in the image are correct (ifautomatically detected), verification of whether the image needs to berectified and/or aligned, a display of the rectified and/or alignedimage if correction is required, a verification of any sidingmeasurements calculated using pixel-to-length ratios, a verification ofthe identified siding products, etc. Siding identification system 10 mayfind reference object dimensions and prompt a user for their knownvalues or allow a user to select and specify these measurements withinthe image. In such embodiments, siding identification system 10determines a pixel-to-length ratio of one or more lengths of referenceobject 134 without user prompts and/or user intervention (other than theentry of the physical dimensions of the reference object).

In accordance with an exemplary embodiment of the present disclosure,siding identification system 10 may additionally send color informationobtained from the image that indicates the desired siding product colorto be matched. A color matching system that may be implemented for thispurpose is described in patent application Ser. No. 14/176,734, entitled“System and Method for Automatically Identifying and Matching a Color ofa Structure's External Surface,” concurrently filed herewith, which isfully incorporated by reference in its entirety. In accordance withcolor matching embodiments, siding product reference database 30 storesthis color information, and siding identification engine 12 utilizes thecolor information together with the length measurements to furthernarrow the pool of identified matching siding products by color beforereturning the results to client device 14.

FIG. 6A is an example screenshot 150 that illustrates an image of asiding sample captured from a severe angle in accordance with anexemplary embodiment of the present disclosure. Using FIG. 5 as anexample, if a user were to capture the siding sample image at an angle,reference object 134 and the siding sample profile would be positionedat an angle. In such a case, using the captured image to determine apixel-to-length ratio associated with the dimensions of the referenceobject 134 will result in distorted measurements. Therefore, embodimentsof the present disclosure correct for such perspective errors.

FIG. 6B is an example screenshot 160 that illustrates an image of asiding sample after performing image rectification on the image of thesiding sample in FIG. 6A in accordance with an exemplary embodiment ofthe present disclosure. To correct for perspective errors, sidingidentification system 10 utilizes image rectification techniques. Aswill be appreciated by those of ordinary skill in the relevant art(s),once the image is detected, a corresponding homography of the image maybe computed. For example, projective spaces associated with thereference object and lines within the siding sample image may bedetermined and associated matrix transformations may be applied to theprojective spaces to provide image rectification. In other words, once areference object within a siding sample is rectified, the resultingrectified image compensates for perspective error. As shown in FIG. 6B,once rectified, the reference object image and image are transformed asif taken from a straight on perspective (i.e., perpendicular to thereference object plane). FIGS. 6A and 6B illustrate an extreme exampleof image rectification for the purposes of explanation. In practice, itis preferable for a user to generally make a good attempt to record theimage in a straight on perspective initially, with the imagerectification providing a refined image to compensate for any usererror.

Additionally or alternatively, embodiments of the present disclosureinclude image alignment techniques to align both the reference objectand one or more edges of the siding image such that they are bothaligned with a horizontal-vertical plane. Using FIG. 5 as an example,the horizontal and vertical edges of reference object 134 are aligned inthe horizontal-vertical plane. However, a user may not initially affixreference object 134 to the siding sample in this way. Furthermore, theimage may be taken from a straight on perspective but skewed such thatthe image is tilted with respect to the horizontal and vertical axes.Since pixels are typically square in shape, edges of reference object134 and the siding sample taken at an angle will be represented by adifferent number of pixels compared to a number of pixels constituting acompletely horizontal or vertical edge length. As a result, if edges ofreference object 134 and the siding sample are not aligned with oneanother and the horizontal-vertical plane, the pixel-to-ratiocalculations from a misaligned reference object and/or skewed image mayresult in less accurate measurements.

Therefore, to correct for such alignment errors, embodiments of thepresent disclosure use image alignment techniques to compensate forimage skew. In accordance with such embodiments, line detection isutilized to measure the angles of lines within the image and thencorrect for skew via coordinate transformations. As will be appreciatedby those of skill in the relevant art(s), examples of line detectiontechniques and transformations include Hough and Radon transforms. Oncethe alignment techniques are applied, reference object 134 and sidingsample 91 are aligned with one another and the horizontal-verticalplace. As a result, pixel-to-length ratios and subsequent calculationsare performed based on a true pixel representation in a singledimension, increasing the accuracy of these measurements.

FIG. 7 illustrates a siding identification method 700 in accordance withan exemplary embodiment of the present disclosure. In variousembodiment, siding identification method 700 is implemented as a part ofsiding identification system 10 as shown in FIG. 1.

Method 700 begins at block 702, at which a communication devicedetermines a pixel-to-length ratio between a number of pixels disposedover a predetermined length of a reference object within an image of asiding sample and the predetermined length of the reference object. Thiscommunication device could include, for example, client device 14 asshown in FIG. 1, and this image could include, for example, screenshotimage 130 as illustrated in FIG. 5.

At block 704, the communication device determines a first and seconddistance between respective first and second pairs of points within theimage corresponding to respective first and second length measurementsof the siding sample. In another embodiment, one pair of points and onelength measurement is determined. The first and second lengthmeasurements could include, for example, face and profile lengths asillustrated in FIG. 5.

At block 706, the communication device determines a first and secondnumber of pixels disposed between the first and second pair of points,respectively. In another embodiment, one number of pixels is determinedcorresponding to one pair of points.

At block 708, the communication device determines the first lengthmeasurement based on the pixel-to-length ratio and the first number ofpixels and the second length measurement based on the pixel-to-lengthratio and the second number of pixels. In another embodiment, one lengthmeasurement is determined based on the pixel-to-length ratio and thefirst number of pixels.

At block 710, a decision is made whether to take additionalmeasurements. In accordance with an embodiment, block 710 may beskipped, for example, if an automatic identification system is used anda predetermined number of measurements are to be made. Block 710 may beperformed, for example, in a semi-automatic identification systemwhereby a user is prompted after one or more measurements have been madeto determine if more measurements are desired. If more measurements areto be made, method 700 continues back to block 704 to obtain additionalpairs of measurement points. If no additional measurements are to bemade, method 700 continues to block 712.

At block 712, the communication device identifies a siding productassociated with the first and second length measurements. In accordancewith an embodiment, the communication device, such as client device 14,for example, sends the first and second length measurements to anothercommunication device, such as siding identification engine 12, forexample. In accordance with such embodiments, siding identificationengine 12 identifies the siding product and sends these identifiedproducts back to the first communication device.

The Figures and accompanying description provided throughout thisdisclosure have been provided largely in reference to siding samples.The embodiments as described throughout this disclosure are equallyapplicable to siding that is installed and to siding samples that havebeen removed from a structure. For example, an agent may use any of thedescribed embodiments to capture an image of a portion of installedsiding that is undamaged and a reference object, and identifying sidingproducts are matched based on such an image.

The following additional considerations apply to the foregoingdiscussion. Throughout this specification, plural instances mayimplement components, operations, or structures described as a singleinstance. Although individual operations of one or more methods areillustrated and described as separate operations, one or more of theindividual operations may be performed concurrently, and nothingrequires that the operations be performed in the order illustrated.Structures and functionality presented as separate components in exampleconfigurations may be implemented as a combined structure or component.Similarly, structures and functionality presented as a single componentmay be implemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter of the present disclosure.

Additionally, certain embodiments are described herein as includinglogic or a number of components or modules. Modules may constituteeither software modules (e.g., code stored on a machine-readable medium)or hardware modules. A hardware module is tangible unit capable ofperforming certain operations and may be configured or arranged in acertain manner. In example embodiments, one or more computer systems(e.g., a standalone, client or server computer system) or one or morehardware modules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In some cases, a hardware module may include dedicated circuitry orlogic that is permanently configured (e.g., as a special-purposeprocessor, such as a field programmable gate array (FPGA) or anapplication-specific integrated circuit (ASIC)) to perform certainoperations. A hardware module may also include programmable logic orcircuitry (e.g., as encompassed within a general-purpose processor orother programmable processor) that is temporarily configured by softwareto perform certain operations. It will be appreciated that the decisionto implement a hardware module in dedicated and permanently configuredcircuitry or in temporarily configured circuitry (e.g., configured bysoftware) may be driven by cost and time considerations.

Accordingly, the term hardware should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which hardwaremodules are temporarily configured (e.g., programmed), each of thehardware modules need not be configured or instantiated at any oneinstance in time. For example, where the hardware modules comprise ageneral-purpose processor configured using software, the general-purposeprocessor may be configured as respective different hardware modules atdifferent times. Software may accordingly configure a processor, forexample, to constitute a particular hardware module at one instance oftime and to constitute a different hardware module at a differentinstance of time.

Hardware and software modules can provide information to, and receiveinformation from, other hardware and/or software modules. Accordingly,the described hardware modules may be regarded as being communicativelycoupled. Where multiple of such hardware or software modules existcontemporaneously, communications may be achieved through signaltransmission (e.g., over appropriate circuits and buses) that connectthe hardware or software modules. In embodiments in which multiplehardware modules or software are configured or instantiated at differenttimes, communications between such hardware or software modules may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware or software moduleshave access. For example, one hardware or software module may perform anoperation and store the output of that operation in a memory device towhich it is communicatively coupled. A further hardware or softwaremodule may then, at a later time, access the memory device to retrieveand process the stored output. Hardware and software modules may alsoinitiate communications with input or output devices, and can operate ona resource (e.g., a collection of information).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods or routines described herein may be at leastpartially processor-implemented. For example, at least some of theoperations of a method may be performed by one or processors orprocessor-implemented hardware modules. The performance of certain ofthe operations may be distributed among the one or more processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment or as a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a SaaS.For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., application program interfaces(APIs).)

The performance of certain of the operations may be distributed amongthe one or more processors, not only residing within a single machine,but deployed across a number of machines. In some example embodiments,the one or more processors or processor-implemented modules may belocated in a single geographic location (e.g., within a homeenvironment, an office environment, or a server farm). In other exampleembodiments, the one or more processors or processor-implemented modulesmay be distributed across a number of geographic locations.

Some portions of this specification are presented in terms of algorithmsor symbolic representations of operations on data stored as bits orbinary digital signals within a machine memory (e.g., a computermemory). These algorithms or symbolic representations are examples oftechniques used by those of ordinary skill in the data processing artsto convey the substance of their work to others skilled in the art. Asused herein, an “algorithm” or a “routine” is a self-consistent sequenceof operations or similar processing leading to a desired result. In thiscontext, algorithms, routines and operations involve physicalmanipulation of physical quantities. Typically, but not necessarily,such quantities may take the form of electrical, magnetic, or opticalsignals capable of being stored, accessed, transferred, combined,compared, or otherwise manipulated by a machine. It is convenient attimes, principally for reasons of common usage, to refer to such signalsusing words such as “data,” “content,” “bits,” “values,” “elements,”“symbols,” “characters,” “terms,” “numbers,” “numerals,” or the like.These words, however, are merely convenient labels and are to beassociated with appropriate physical quantities.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,condition A or B is satisfied by any one of the following: A is true (orpresent) and B is false (or not present), A is false (or not present)and B is true (or present), and both A and B are true (or present).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the description. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs forproviding an interface for inspecting indoor and outdoor map datathrough the disclosed principles herein. Thus, while particularembodiments and applications have been illustrated and described, it isto be understood that the disclosed embodiments are not limited to theprecise construction and components disclosed herein. Variousmodifications, changes and variations, which will be apparent to thoseskilled in the art, may be made in the arrangement, operation anddetails of the method and apparatus disclosed herein without departingfrom the spirit and scope defined in the appended claims.

What is claimed is:
 1. A computer-implemented method for identifyingsiding products, comprising: adjusting, by one or more processors, anorientation of a reference object within an image of a siding sample toalign edges of the reference object with edges of the siding sample;determining, by one or more processors, a pixel-to-length ratiorepresentative of a ratio between a first number of pixels disposed overa length of the reference object and the length of the reference object;determining, by one or more processors, a first siding measurement basedupon the pixel-to-length ratio and a second number of pixels disposedover a first portion of the image of the siding sample corresponding tothe first siding measurement; and identifying, by one or moreprocessors, one or more siding products associated with the first sidingmeasurement.
 2. The method of claim 1, further comprising: determining,by one of more processors, whether the orientation of the referenceobject needs to be adjusted such that the edges of the reference objectare aligned with edges of the siding sample; and adjusting theorientation of the reference object only when it is determined that theorientation needs to be adjusted.
 3. The method of claim 1, furthercomprising: rectifying, by one or more processors, the image of thesiding sample so the perspective of the siding sample faces forward ifthe reference object includes the perspective distortion.
 4. The methodof claim 1, wherein the act of adjusting the orientation of thereference object comprises: adjusting the orientation of the referenceobject and the siding sample in the image such that respectivehorizontal and vertical edges of each of the reference object and thesiding sample are aligned with edges corresponding to ahorizontal-vertical plane.
 5. The method of claim 1, further comprising:determining a second siding measurement based upon the pixel-to-lengthratio and a third number of pixels disposed over a second portion of theimage of the siding sample corresponding to the second sidingmeasurement, and wherein the act of identifying the one or more sidingproducts comprises: identifying the one or more siding productsassociated with the first and the second siding measurement.
 6. Themethod of claim 5, wherein the first siding measurement corresponds to aface size of the siding sample, and wherein the second sidingmeasurement corresponds to a profile size of the siding sample.
 7. Themethod of claim 6, wherein the act of identifying the one or more sidingproducts comprises: identifying the one or more siding products fromsiding products stored in the siding product reference database having aface size and a profile size matching the face size and the profile sizeof the siding sample.
 8. The method of claim 1, further comprising:detecting, by one or more processors, the reference object using atleast one of: a Scale-Invariant Feature Transform (SIFT); a Speeded UpRobust Features (SURF); a Radon transform; or a Hough transform.
 9. Anon-transitory, tangible computer-readable medium storing machinereadable instructions that, when executed by a processor, cause theprocessor to: adjust an orientation of a reference object within animage of a siding sample to align edges of the reference object withedges of the siding sample; determine a pixel-to-length ratiorepresentative of a ratio between a first number of pixels disposed overa length of the reference object and the length of the reference object;determine a first siding measurement based upon the pixel-to-lengthratio and a second number of pixels disposed over a first portion of theimage of the siding sample corresponding to the first sidingmeasurement; and identify one or more siding products associated withthe first siding measurement.
 10. The non-transitory, tangiblecomputer-readable medium of claim 9, further storing instructions that,when executed by the processor, cause the processor to: determinewhether the orientation of the reference object needs to be adjustedsuch that the edges of the reference object are aligned with edges ofthe siding sample; and adjust the orientation of the reference objectonly when it is determined that the orientation needs to be adjusted.11. The non-transitory, tangible computer-readable medium of claim 9,further storing instructions that, when executed by the processor, causethe processor to: rectifying the image of the siding sample so theperspective of the siding sample faces forward if the reference objectincludes the perspective distortion.
 12. The non-transitory, tangiblecomputer-readable medium of claim 9, wherein the instructions to adjustthe orientation of the reference object further include instructions,that when executed by the processor, cause the processor to: adjust theorientation of the reference object and the siding sample in the imagesuch that respective horizontal and vertical edges of each of thereference object and the siding sample are aligned with edgescorresponding to a horizontal-vertical plane.
 13. The non-transitory,tangible computer-readable medium of claim 9, wherein the instructionsto determine the siding measurement further include instructions, thatwhen executed by the processor, cause the processor to: determine asecond siding measurement based upon the pixel-to-length ratio and athird number of pixels disposed over a second portion of the image ofthe siding sample corresponding to the second siding measurement, andwherein the instructions to identify the one or more siding productsfurther include instructions, that when executed by the processor, causethe processor to: identify the one or more siding products associatedwith the first and the second siding measurement.
 14. Thenon-transitory, tangible computer-readable medium of claim 13, whereinthe first siding measurement corresponds to a face size of the sidingsample, and wherein the second siding measurement corresponds to aprofile size of the siding sample.
 15. The non-transitory, tangiblecomputer-readable medium of claim 14, wherein the instructions toidentify the one or more siding products further include instructions,that when executed by the processor, cause the processor to: identifythe one or more siding products from siding products stored in thesiding product reference database having a face size and a profile sizematching the face size and the profile size of the siding sample. 16.The non-transitory, tangible computer-readable medium of claim 9,further storing instructions that, when executed by the processor, causethe processor to: detect the reference object using at least one of: aScale-Invariant Feature Transform (SIFT); a Speeded Up Robust Features(SURF); a Radon transform; or a Hough transform.
 17. A siding productidentification system, comprising: a mobile device configured to (i)adjust an orientation of a reference object within an image of a sidingsample to align edges of the reference object with edges of the sidingsample, (ii) determine a pixel-to-length ratio representative of a ratiobetween a first number of pixels disposed over a length of the referenceobject and the length of the reference object, (iii) facilitateselection by a user of a first portion of the image of the siding samplecorresponding to a first siding measurement, and (iv) determine thefirst siding measurement based upon the pixel-to-length ratio and asecond number of pixels disposed over the first portion of the image ofthe siding sample selected by the user; and a siding productidentification engine configured to identify one or more siding productsassociated with the first siding measurement.
 18. The siding productidentification system of claim 17, wherein the mobile device is furtherconfigured to: determine whether the orientation of the reference objectneeds to be adjusted such that the edges of the reference object arealigned with edges of the siding sample; and adjust the orientation ofthe reference object only when it is determined that the orientationneeds to be adjusted.
 19. The siding product identification system ofclaim 17, wherein the mobile device is further configured to: performimage rectification on the image of the siding sample so the perspectiveof the siding sample faces forward if the reference object includes theperspective distortion.
 20. The siding product identification system ofclaim 17, wherein the mobile device is further configured to: adjust theorientation of the reference object and the siding sample in the imagesuch that respective horizontal and vertical edges of each of thereference object and the siding sample are aligned with edgescorresponding to a horizontal-vertical plane.
 21. The siding productidentification system of claim 17, wherein the mobile device is furtherconfigured to: determine a second siding measurement based upon thepixel-to-length ratio and a third number of pixels disposed over asecond portion of the image of the siding sample corresponding to thesecond siding measurement, and wherein the siding product identificationengine is further configured to identify the one or more siding productsassociated with the first and the second siding measurement.
 22. Thesiding product identification system of claim 21, wherein the firstsiding measurement corresponds to a face size of the siding sample, andwherein the second siding measurement corresponds to a profile size ofthe siding sample.
 23. The siding product identification system of claim22, wherein the siding product identification engine is furtherconfigured to: receive data representative of the face size and theprofile size of the siding sample from the mobile device; identify theone or more siding products from siding products stored in the sidingproduct reference database having a face size and a profile sizematching the face size and the profile size of the siding sample; andsend data representative of the identified one or more siding productsto the mobile device.
 24. The siding product identification system ofclaim 17, wherein the mobile device is further configured to detect thereference object using at least one of: a Scale-Invariant FeatureTransform (SIFT); a Speeded Up Robust Features (SURF); a Radontransform; or a Hough transform.